Motion vector conversion method and conversion apparatus

ABSTRACT

The invention provides a motion vector conversion method by which the cording efficiency in image coding of MPEG4 in an image information conversion method can be augmented. In the motion vector conversion method for an image information conversion method wherein a bit stream representative of interlaced scanned image compression information of MPEG2 is inputted and a bit stream representative of progressively scanned image compression information of MPEG4 is outputted, 16×16 motion vectors of MPEG2 of the inputted bit stream representative of image compression information of MPEG2 are accepted successively, and 8×8 motion vectors of MPEG4 and 16×16 motion vectors of MPEG4 are produced successively based on the 16×16 motion vectors of MPEG2. Every other one of I frames and P frames of the bit stream of MPEG2 is dropped to produce a bit stream of MPEG4 of a reduced frame rate and a low bit rate.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based upon and claims the benefit of priorityfrom the related Japanese Patent Application No. 2000-310836, filed Oct.11, 2000, the entire contents of which are incorporated herein byreference.

[0002] The present application also contains the subject matter relatedto concurrently filed U.S. Patent Applications by Takahashi, et al.entitled “Motion Vector Conversion Method and Conversion Apparatus”,based upon and claims the benefit of priority from the related JapanesePatent Application No. 2000-312309.

[0003] The related applications are assigned to the same assignee as thepresent application.

BACKGROUND OF THE INVENTION

[0004] This invention relates to a motion vector conversion method and amotion vector conversion apparatus for use with an image informationconversion method and an image information conversion apparatus whereinthe MPEG system is used to receive, through a satellite broadcast, acable TV or network media such as the Internet, or process, on suchstorage media as an optical or magnetic disk or a flash memory, a bitstream representative of image information compressed by orthogonaltransform such as discrete cosine transform and motion compensation.

[0005] In recent years, an apparatus which complies with the MPEG systemwherein image information is treated as digital data and compressed byorthogonal transform such as discrete cosine transform and motioncompensation making use of the redundancy unique to the imageinformation in order to transmit and store the information with a highefficiency has been and is being popularized in both of informationdistribution of broadcasting stations and so forth and informationreception in general homes.

[0006] Particularly, the MPEG2 (ISO/IEC 13818-2) is defined as a generalpurpose image coding system and is a standardized system which coversboth of interlaced scanned images and progressively scanned images aswell as standard resolution images and high resolution images. It isestimated that the MPEG2 is used for a wide variety of applications forprofessional use and for consumer use also in the future. The use of thecompression system of MPEG2 can achieve a high compression ratio and agood picture quality, for example, for both of interlaced scanned imagesof a standard resolution having 720×480 pixels or interlaced scannedimages of a high resolution having 1,920×1,088 pixels at 4 to 8 Mbps byallocating a code rate (bit rate) of 18 to 22 Mbps.

[0007] Although MPEG2 is directed principally to high picture qualitycoding suitable for broadcasting but is not ready for a lower code rate(bit rate) than that of MPEG1, that is, not ready for a coding system ofa higher compression ratio. It is considered that the need for such acoding system as just described increases in the future a popularizationof portable terminals proceeds. In order to cope with the need,standardization of the MPEG4 coding system has been performed. Thestandards for the MPEG4 image coding system have been approved asinternational standards as ISO/IEC 14496-2 in December 1998.

[0008] Meanwhile, it is requested to convert a bit stream whichrepresents image compression information of MPEG2 coded once for digitalbroadcasting into another bit stream representative of image compressioninformation OF mpeg4 of a lower code rate (bit rate) which is moresuitable for processing on a portable terminal or the like.

[0009] An example of a related art image information conversionapparatus which satisfies the request is shown in FIG. 1. Referring toFIG. 1, the image information conversion apparatus shown converts a bitstream representative of image compression information of MPEG2 inputtedthereto into another bit stream representative of image compressioninformation of MPEG4. In particular, the inputted bit streamrepresentative of image compression information of MPEG2 is decoded byan MPEG2 image information decoding apparatus 1. The decoded imagesignal is transmitted to a resolution frame rate conversion apparatus 2,by which it is converted into an image signal having an arbitrarydifferent resolution and frame rate. The image signal obtained by theconversion is inputted to an MPEG4 image information coding apparatus 3,by which it is coded into and outputted as a bit stream representativeof image compression information of MPEG4.

[0010] In the related art image information conversion apparatus, asseen in FIG. 1, the image signal decoded in accordance with the MPEG2decoding system is coded by the MPEG4 image information codingapparatus, and a bit stream representative of image compressioninformation of MPEG4 is outputted. The MPEG2 image information decodingapparatus 1 may be configured such that it performs a decoding processfor both of horizontal and vertical direction components using alleighth order DCT (discrete cosine transform) coefficients of the bitstream representative of the inputted image compression information ofMPEG2. However, the MPEG2 image information decoding apparatus 1 may beconfigured otherwise such that it performs a decoding process whereinall of eighth order coefficients in the vertical direction are used butonly four lower frequency ones of eighth order coefficients in thehorizontal direction are used (the decoding process is hereinafterreferred to as 4×8 down decoding) or another decoding process whereinonly four lower frequency ones of eighth order coefficients in both ofthe horizontal direction and the vertical directions are used (thedecoding process is hereinafter referred to as 4×4 down decoding) inorder to reduce the arithmetic operation amount and the video memorycapacity and simplify down sampling processing in the following stagewhile suppressing the picture quality deterioration to the minimum.

[0011] According to such a related art method as described above, whenthe MPEG4 image information coding apparatus codes an image signalinputted thereto, the arithmetic operation processing amount fordetecting a motion vector occupies approximately 60 to 70% of the totalarithmetic operation processing amount. This gives rise to such problemsthat real time processing of an image becomes difficult, that a timedelay occurs and that a large apparatus scale is required.

[0012] As a countermeasure for solving such problems as just described,the inventors have proposed an image information conversion apparatusshown in FIG. 2. Particularly, the inventors have filed the followingapplications for patent in Japan relating to the image conversionapparatus shown in FIG. 2.

[0013] Japanese Patent Application No. 2000-129002 “Motion VectorConversion Apparatus and Method”: This application discloses a techniquewherein information of each of macro blocks of MPEG2 is used to selectan motion vector of MPEG 2 of that one of macro blocks of MPEG2 whichexhibits the highest coding efficiency to produce a motion vector ofMPEG4.

[0014] Japanese Patent Application No. 2000-132915 “Motion VectorConversion Apparatus and Method”: This application discloses a techniquewherein information of each of macro blocks is used to select, based onlengths of motion vectors, a motion vector of MPEG2 of that one of macroblocks of MPEG2 which exhibits the highest coding efficiency to producea motion vector of MPEG4.

[0015] Japanese Patent Application No. 2000-191616 “Motion VectorConversion Apparatus and Method”: This application discloses a techniquewherein information of each of macro blocks is used to produce a P-VOPmotion vector for an intra-macro block of MPEG2.

[0016]FIGS. 3A and 3B illustrate a correlation between a motion vectorin a bit stream representative of image compression information of MPEG2and a motion vector in a bit stream representative of image compressioninformation of MPEG4 and particularly show images of a current framebefore and after resolution conversion, respectively. When theresolution of an image is converted, a horizontal component of a motionvector from the position in the preceding frame to a position in thecurrent frame after the conversion can be determined from a horizontalcomponent of a motion vector before the resolution conversion and theresolution conversion rate in the horizontal direction of the image. Avertical component of the motion vector after the resolution conversioncan be determined from a vertical component of a motion vector beforethe resolution conversion and the resolution conversion rate in thevertical direction of the image. In other words, the motion vectorbefore the resolution conversion and the motion vector after theconversion have a high correlation. The correlation can be utilized todetermine the motion vector after the resolution conversion from themotion vector before the resolution conversion.

[0017] In particular, the image information conversion apparatus of FIG.2 simply converts a motion vector in a bit stream representative ofimage compression information of MPEG2 inputted thereto into a motionvector of MPEG4 making use of parameters such as a motion vector of amacro block of MPEG2 and a macro block type. In an MPEG4 imageinformation coding apparatus 7, detection of a motion vector is notperformed, but image coding using motion vectors obtained by conversionis performed. As a result, the MPEG4 image information coding apparatus7 does not perform motion detection, and consequently, the processingamount is reduced significantly.

[0018] By performing conversion from a motion vector of MPEG2 into amotion vector of MPEG4 and adopting a parameter used for decoding ofMPEG2 or a parameter after conversion in addition to a motion vector inthis manner, the processing amount of the MPEG4 image information codingapparatus 7 can be reduced. Consequently, the time delay by the MPEG4image information coding apparatus 7 can be reduced.

[0019] The bit stream representative of the inputted image compressioninformation of MPEG2 illustrated in FIGS. 3A and 3B undergoes decodingprocessing by an MPEG2 image information decoding apparatus 4 of FIG. 2,and a bit stream representative of image compression information ofMPEG4 is outputted. The MPEG2 image information decoding apparatus 4 maybe configured such that it performs a decoding process for both ofhorizontal and vertical direction components using all eighth order DCTcoefficients of the bit stream representative of the inputted imagecompression information of MPEG2. However, the MPEG2 image informationdecoding apparatus 4 may be configured otherwise such that it performs4×8 down decoding or 4×4 down decoding in order to reduce the arithmeticoperation amount and the video memory capacity and simplify downsampling processing in the following stage while suppressing the picturequality deterioration to the minimum. The image signal outputted fromthe MPEG2 image information decoding apparatus 4 is sent to a resolutionframe rate conversion apparatus 5, in which resolution and frame rateconversion are performed and then an image signal having a resolutionsuitable for image coding of MPEG4 is produced with an image sizeadjustment flag inputted thereto from the outside. The thus producedimage signal is outputted from the resolution frame rate conversionapparatus 5.

[0020] More particularly, the resolution frame rate conversion apparatus5 first performs resolution conversion of the image signal inputtedthereto from the MPEG2 image information decoding apparatus 4 by meansof a resolution frame converter. Here, an example of resolutionconversion wherein the resolution is reduced to ½ for both of thevertical and horizontal directions is described. As seen in FIG. 4, inthe down sampling in the vertical direction, first fields or secondfields of inputted interlaced scanned images are extracted to convertthe images into progressively scanned images. In the down sampling inthe horizontal direction, a down sampling filter is used to convert eachimage into an image of a ½ resolution. Further, in order to achieve alow bit rate, not only compression by resolution conversion isperformed, but also first fields or second fields only of I/P picturesare extracted to lower the frame rate in the temporal direction. Forexample, I, B, B and P pictures of MPEG2 shown in FIG. 4 are convertedinto the first fields of the I and P pictures by resolution and framerate conversion. The images obtained by the resolution and frameconversion include a number of pixels arranged in numbers of rows andcolumns equal to multiples of 16 so that they can be coded in accordancewith a picture coding method of MPEG4. To this end, a circuit forsupplementing or removing pixels is employed to supplement or removepixels in accordance with an image size adjustment flag inputted fromthe outside.

[0021] The image size adjustment flag is inputted from the outside ofthe resolution frame rate conversion apparatus 5 and is used fordiscrimination of whether pixels should be supplemented to or removedfrom an image when the pixel number in the rows or columns of the imageis not a multiple of 16.

[0022] A process for an image with the image size adjustment flag isdescribed with reference to FIG. 5. If it is assumed that the resolutionof an image outputted from the MPEG2 image information decodingapparatus 4 is m×n pixels, then while both of m and n are multiples of16, m/2 and n/2 by down sampling to ½ in the vertical and horizontaldirections are multiples of 16 or have the remainder of 8 pixels whenthey are divided by 16. Where both of m/2 and n/2 are multiples of 16,the image complies with the MPEG4 coding system, and therefore, noprocessing is performed for the image. In any other case, the image doesnot comply with the MPEG4 coding system, and therefore, it is necessaryto process the image with the image size adjustment flag. The image sizeadjustment flag provides two choices of supplementation and removal ofpixels. When m/2 or n/2 is divided by 16, if the remainder is 8 pixels,then if removal of pixels is selected, then the remaining 8 pixels areremoved. In other words, the resulting output image has (m/2−8) or(n/2−8) rows or columns. On the other hand, if supplementation of pixelsis selected, then 8 pixels produced newly, 8 pixels copied from theoriginal image or 8 pixels suitable for the image are added to the topor the bottom of the rows or columns of the image. Therefore, the outputimage has (m/2+8) or (n/2+8) rows or columns. As a result, thehorizontal and vertical resolutions of the image after the conversionbecome multiples of 16, and an image having a size compliant with thecoding system of MPEG4 is outputted.

[0023] Referring back to FIG. 2, the inputted bit stream representativeof image compression information of MPEG2 undergoes variable lengthdecoding by the MPEG2 image information decoding apparatus 4. Further,macro block motion vectors only of P pictures and other parameters suchas the macro block type are extracted from the bit stream by the MPEG2image information decoding apparatus 4 and transmitted to a motionvector conversion apparatus 6.

[0024] A motion vector conversion method by the motion vector conversionapparatus 6 is described with reference to FIGS. 6A and 6B. Each ofsquares defined by solid horizontal and vertical lines in FIGS. 6A and6B indicates a macro block. FIG. 6A shows an image outputted from theMPEG2 image information decoding apparatus 4, that is, an image beforeresolution conversion. FIG. 6B shows an image obtained by converting thevertical and horizontal resolutions of the image of FIG. 6A into ½ bymeans of the resolution frame rate conversion apparatus 5. For example,a macro block of 16×16 pixels (hereinafter referred to as 16×16 macroblock) indicated with slanting lines at the left top corner of the imageof FIG. 6A before the conversion is converted into a block of 8×8 pixels(hereinafter referred to as 8×8 block) indicated with slanting lines atthe left top corner of the image of FIG. 6B. Further, four 16×16 macroblocks screened in FIG. 6A are individually converted, by resolutionconversion, into four 8×8 blocks as screened in FIG. 6B, and one 16×16macro block is formed from the four 8×8 macro blocks. Since thecorrelation between motion vectors before and after the resolutionconversion is high, a motion vector of an 8×8 block after the conversioncan be determined from a motion vector of a 16×16 macro block before theconversion. Further, a 16×16 motion block is determined from four 8×8motion vectors. Consequently, four 8×8 motion vectors and one 16×16motion vectors for use for coding in accordance with the image codingsystem of MPEG4 are produced.

[0025] A principle of operation of motion vector conversion is describedwith reference to FIG. 7 which shows a detailed configuration of themotion vector conversion apparatus 6 of the image information conversionapparatus of FIG. 2. Such parameters as a motion vector and an imagesize in the inputted bit stream representative of image compressioninformation of MPEG2 are used to produce an 8×8 motion vector beforemodification by an MPEG2 16×16 motion vector to 8×8 motion vectorconversion apparatus 8.

[0026] A principle of operation of the MPEG2 16×16 motion vector to 8×8motion vector conversion apparatus 8 is described with reference to aflow chart of FIG. 8. The MPEG2 16×16 motion vector to 8×8 motion vectorconversion apparatus 8 operates in the following manner based on amotion vector and a macro block type of an inputted macro block ofMPEG2. Since a bit stream of interlaced scanned image compressioninformation of MPEG2 usually uses a frame structure, a conversion methodwhich only includes processing of a bit stream of a frame structure isdescribed here.

[0027] In step ST-1, the MPEG2 16×16 motion vector to 8×8 motion vectorconversion apparatus 8 discriminates, based on the motion vector and themacro block type of the inputted macro block of MPEG2, which one of anintra-macro block, a skip macro block and an inter-macro block the macroblock is. If the macro block is an intra-macro block, then it is assumedthat an 8×8 block after resolution conversion of the intra-macro blockof MPEG2 has a motion vector. As processing for the intra-macro block ofMPEG2, in step ST-2, the motion vector of the 8×8 block is set to 0first, and then an intra-mode flag is provided in order to allowprocessing by a MPEG2 intra-macro block motion vector modifier 11 to beperformed. In MPEG2, when the macro block is an intra-macro block, anintra-mode flag is set.

[0028] If the macro block otherwise is a skip macro block, then themotion vector of each block is set to 0.

[0029] A concept of motion vector conversion where the macro block is aninter-macro block and the image has a frame structure and frameprediction is used is described with reference to FIGS. 9A and 9B. FIG.9A shows an image before resolution conversion while FIG. 9B shows theimage after the resolution conversion. As described hereinabove withreference to FIGS. 3A and 3B, a horizontal component of a motion vectorafter conversion is determined from a horizontal component of a motionvector before the conversion and a resolution conversion rate in thehorizontal direction of the image. A vertical component of the motionvector is determined from a vertical component of the motion vectorbefore the conversion and a resolution conversion rate in the verticaldirection of the image. In particular, when the resolution in thehorizontal direction is converted into ½, also the horizontal componentof the motion vector after the conversion becomes ½ that before theconversion. When the resolution in the vertical direction is convertedinto ½, also the vertical component of the motion vector after theconversion becomes ½ that before the conversion.

[0030] The motion vector (8, 12) before conversion illustrated in FIG.9A changes to a motion vector (4, 6) after the conversion illustrated inFIG. 9B. It is to be noted that, in FIGS. 9A and 9B, the distancebetween middle values (half pixels) of integer pixels is representedby 1. In the image before the resolution conversion shown in FIG. 9A,each dark circle indicates the position of an integer pixel, and eachrhomb indicates the position of a half pixel. In the image after theresolution conversion shown in FIG. 9B, a half pixel is indicated by ablank circle. As can be seen from FIGS. 9A and 9B, a motion vectorindicated at the position of an integer pixel before conversion isindicated at the position of an integer pixel or a half pixel after theconversion, but a motion vector indicated at the position of a halfpixel before the conversion does not have a pixel referred to therebyafter the resolution conversion. Therefore, if a motion vector beforeconversion indicates the position of a half pixel, then also the motionvector after conversion is changed so as to indicate the position of ahalf pixel of a predictive image. Since a decoded image signaloriginally includes some distortion arising from quantization, if it isused as it is as a predictive image, then the prediction efficiency isdeteriorated and may sometimes cause picture quality deterioration. Inorder to reduce such prediction efficiency deterioration, a half pixelaccuracy wherein pixels on a reference screen which corresponds to alow-pass filter are linearly interpolated at 1:1 is sometimes selectedso that the picture quality deterioration may be prevented. Accordingly,also in coding in accordance with the image coding system of MPEG4, inorder to augment the prediction efficiency and prevent the picturequality deterioration, if a motion vector of MPEG4 indicates theposition of a half pixel, then also upon conversion into a format ofMPEG4, it is converted so that the motion vector may indicate theposition of a half pixel. A corresponding relationship between motionvectors before and after conversion is illustrated in a table of FIG.10.

[0031]FIGS. 11A and 11A illustrate a concept of motion vector conversionwhere an image has a frame structure and field prediction, particularlyfirst field prediction, is used. A horizontal component of a motionvector is processed in a similar manner as in the image of FIGS. 10A and10B. For the vertical direction, the first fields are extracted to allowconversion of the resolution into ½. Since also the prediction used isthe first field prediction, a motion vector before conversion becomes amotion vector after the conversion as it is.

[0032] A concept of motion vector conversion where an image has a framestructure and second field prediction is used is illustrated in FIGS.12A, 12B and 13. When resolution conversion is performed, since only thefirst field is extracted, pixel values of the first field are used as areference image after conversion. Therefore, temporal and spatialmodification of a motion vector is performed so that pixel values of thesecond field used as a predictive picture in MPEG2 are used for firstfield prediction after resolution conversion. FIGS. 12A and 12Billustrate a technique of spatial modification used to convert pixelvalues of second field prediction approximately into pixel values offirst field prediction. FIG. 12A illustrates a motion vector beforeresolution conversion while FIG. 12B illustrates a motion vector afterthe resolution conversion. In particular, “1” is added to a verticalcomponent of a motion vector. As can be seen from FIGS. 12A and 12B, if“1” is added to a vertical component of a motion vector determined bysecond field prediction, then the second field is shifted by one rowdistance so that it comes to a spatial position similar to that of thefirst field and the motion vector becomes a motion vector like a motionvector determined by first field prediction in the space. The expression(1) given below represents a vertical component of a motion vectorMV_(top) where the second field at a spatial position positioned at asimilar position to that of the first field by spatial modification,that is, an approximate first field, is used for prediction:

Vertical component: approximate MV _(top) =MV _(bottom)+1  (1)

[0033] Meanwhile, interlaced scanned image compression information ofMPEG2 exhibits some displacement in time between the first field and thesecond field. Therefore, modification for the time displacement betweenthe first field approximated from the second field and the actual firstfield is performed. FIG. 13 illustrates a temporal positionalrelationship of different fields. Here, if the interval between thefirst field and the second field is represented by 1 and the intervalbetween the second field of an I picture and the first field of a Ppicture is represented by a, then the value a exhibits an odd numberlike 1, 3, 5, 7, . . . It is to be noted that, where a is 1, picturesare I, P, P, P, . . . pictures. A time modified motion vector MV′ isgiven by the following expression (2):

Vertical component: MV′={(a+1)/a}approximate MV _(top)  (2)

[0034] By substituting the expression (1) into the expression (2), thevertical component of the motion vector after the conversion is given bythe following expression (3):

Vertical component: MV′={(a+1)/a}(MV _(bottom)+1)  (3)

[0035] It is to be noted that the horizontal component of the motionvector after the conversion can be determined by multiplying the motionvector before the conversion by (a+1)/a, performing temporalmodification and then performing the calculation illustrated in thetable of FIG. 10.

[0036] Spatial modification may be performed for the vertical componentof the motion vector after the temporal modification is performed, ifneeded. The vertical component of the motion vector MV′ in this instanceis given by the expression (4) given below. It is to be noted that, ifspatial and temporal modification is performed for the horizontalcomponent (time modification is performed after spatial modification isperformed), then the horizontal component exhibits a similar value tothat obtained when temporal and spatial modification is performed(spatial modification is performed after temporal modification isperformed).

Vertical component: MV′={(a+a)/1}MV _(bottom)+1  (4)

[0037] The difference between the expression (3) and the expression (4),that is, the difference between the vertical components of the motionvector when spatial and temporal modification is performed and whentemporal and spatial modification is performed, is 1/a. Accordingly,since the influence of the difference differs depending upon the valueof a, modification methods in two cases wherein a is equal 1 and whereina is 3, 5, 7, . . . are described.

[0038] First, a modification method where a=1 is described. Bysubstituting 1 into a of the expression (3) the vertical component ofthe motion vector is determined as given by the following expression(5):

Vertical component: MV′=2×(MV _(bottom)+1)  (5)

[0039] By substituting 1 into a of the expression (4), the verticalcomponent of the motion vector is determined as given by the followingexpression (6):

Vertical component: MV′=2×(MV _(bottom)−1)  (6)

[0040] As a result, if 0, 1, 2, . . . are substituted into the motionvector MV_(bottom) before the conversion, then such even numbers as 2,4, 6, . . . are obtained as the value according to the expression (5).On the other hand, such odd numbers as 1, 3, 5, . . . are obtained asthe value according to the expression (6). In other words, if spatialand temporal modification is performed, then irrespective of whether themotion vector before the conversion is indicated at the position of aninteger pixel or at the position of a half pixel, the motion vectorafter the conversion comes to the position of a half pixel. Accordingly,in order to cause a motion vector, which is indicated at the position ofan integer pixel before conversion, to come to the position of aninteger pixel also after the conversion, spatial and temporal conversionis performed. On the other hand, in order to cause a motion vector,which is indicated at the position of a half pixel before conversion, tocome to the position of a half pixel also after the conversion, temporaland spatial conversion is performed. In short, spatial modification andtemporal modification are used alternately for motion vectors beforeconversion to convert them into motion vectors after the resolutionconversion or to perform temporal and spatial modification for themotion vectors before the conversion.

[0041] After the motion vector conversion process described above comesto an end, an 8×8 motion vector of MPEG4 before modification isoutputted. The 8×8 motion vector outputted is transmitted to an imagesize adjustment flag based motion vector adjuster 9 (FIG. 7), by whichit is modified with an image size adjustment flag inputted thereto fromthe outside to a motion vector suitable for an image size. The resultingmotion vector is outputted from the image size adjustment flag basedmotion vector adjuster 9.

[0042] Operation of the image size adjustment flag based motion vectoradjuster 9 is described with reference to a flow chart of FIG. 14. Instep ST-11, the image size adjustment flag based motion vector adjuster9 discriminates whether or not both of m/2 and n/2 where the input imagehas a size of m×n pixels are multiples of 16. If both of m/2 and n/2 aremultiples of 16, then the image size adjustment flag based motion vectoradjuster 9 outputs the 8×8 motion vector of MPEG4 outputted from themotion vector conversion apparatus 6 as it is without processing thesame. If m/2 or n/2 is not a multiple of 16, the image size adjustmentflag based motion vector adjuster 9 uses the image size adjustment flaginputted from the outside to discriminate whether or not pixels shouldbe removed in step ST-12. If pixels should be removed, then the imagesize adjustment flag based motion vector adjuster 9 does not output the8×8 motion vector of the eight pixels removed but outputs another 8×8motion vector. If it is discriminated in step ST-12 that pixels shouldnot be removed, then the image size adjustment flag based motion vectoradjuster 9 discriminates in step ST-13 whether or not pixels should besupplemented. If pixels should be supplemented, then the image sizeadjustment flag based motion vector adjuster 9 sets an 8×8 motion vectorof eight pixels supplemented to 0 and outputs the motion vector of 0together with the other inputted 8×8 motion vectors.

[0043] Referring back to FIG. 7 again, the 8×8 motion vector outputtedfrom the image size adjustment flag based motion vector adjuster 9 andsuitable for the image size is converted by an MPEG4 8×8 motion vectorto MPEG4 16×16 motion vector conversion apparatus 10 or an MPEG4 8×8motion vector to MPEG4 16×16 motion vector conversion apparatus 15 shownin FIG. 16.

[0044] The motion vector conversion apparatus 10 of FIG. 7 divides thesum of motion vectors of those of four blocks cooperatively forming amacro block which are converted from a macro block which is not anintra-macro block b the number of those blocks to calculate an averagemotion vector and outputs the average motion vector as a 16×16 motionvector.

[0045] This is described with reference to FIG. 16. As a first method,the MPEG4 8×8 motion vector to MPEG4 16×16 motion vector conversionapparatus 15 selects, from among 8×8 motion vectors of MPEG4 for themacro block produced by an MPEG2 16×16 motion vector to MPEG4 8×8 motionvector conversion apparatus 12, the motion vector produced from that oneof the pertaining macro blocks which is considered to have the highestcoding efficient, and outputs the selected motion vector as a 16×16motion vector of MPEG4.

[0046] The discrimination of the coding efficiency is performed based oninformation of the individual macro blocks stored in a macro blockinformation buffer 14 and representative of image compressioninformation of MPEG2 inputted to the image information conversionapparatus.

[0047] In short, the first method determines that one of four macroblocks which includes the least number of non-zero DCT coefficients hasthe highest coding efficiency. A second method determines that one offour macro blocks which includes the least number of bits allocated toDCT coefficients of brightness components has the highest codingefficiency. A third method determines that one of four macro blockswhich includes the least number of bits allocated to DCT coefficientshas the highest coding efficiency. A fourth method determines that oneof four macro blocks which includes the least total number of bitsallocated to the macro block including motion vectors and so forth hasthe highest coding efficiency. A fifth method determines that one offour macro blocks which has the smallest allocated quantization scalehas the highest coding efficiency. A fifth method determines that one offour macro blocks which has the lowest complexity has the lowest codingefficiency. The complexity X allocated to each macro block is calculatedin accordance with the following expression (7) using the quantizationscale Q allocated to the macro block and the bit number B of the macroblock:

X=Q′B  (7)

[0048] where B may be the bit number allocated to the entire macroblock, or may be the bit number allocated to DCT coefficients or elsemay be the bit number allocated to DCT coefficients allocated tobrightness components.

[0049] Referring back to FIG. 16, as the second method, the MPEG4 8×8motion vector to MPEG4 16×16 motion vector conversion apparatus 15counts, from among 8×8 motion vectors of MPEG4 for the macro blocksproduced by the MPEG2 16×16 motion vector to MPEG4 8×8 motion vectorconversion apparatus 12, that motion vector produced from the macroblock which is considered to have the highest weight twice in anoverlapping relationship, and determines that one of the totaling five8×8 motion vectors which has a length of a middle value has the highestcoding efficiency and selects and outputs the 8×8 motion vector as a16×16 motion vector of MPEG4. Here, for the comparison in length betweenmotion vectors, the sum of the squares of the lengths in the horizontaldirection and the vertical direction is used, but a process ofdetermining a square root is omitted.

[0050] The discrimination of the weight is performed based oninformation of the individual macro blocks of the bit stream stored inthe macro block information buffer 14 and representative of imagecompression information of MPEG2 inputted to the image informationconversion apparatus.

[0051] In particular, a first method determines that one of four macroblocks which includes the least number of non-zero DCT coefficients hasthe highest weight. A second method determines that one of four macroblocks which includes the least number of bits allocated to DCTcoefficients of brightness components has the highest weight. A thirdmethod determines that one of four macro blocks which includes the leastnumber of bits allocated to DCT coefficients has the highest weight. Afourth method determines that one of four macro blocks which includesthe least total number of bits allocated to the macro block includingmotion vectors and so forth has the highest weight. A fifth methoddetermines that one of four macro blocks which has the smallestallocated quantization scale has the highest weight. A fifth methoddetermines that one of four macro blocks which has the lowest complexityhas the lowest weight. The complexity X allocated to each macro block iscalculated in accordance with the following expression (7) using thequantization scale Q allocated to the macro block and the bit number Bof the macro block:

X=Q′B  (8)

[0052] where B may be the bit number allocated to the entire macroblock, or may be the bit number allocated to DCT coefficients or elsemay be the bit number allocated to DCT coefficients allocated tobrightness components.

[0053] Referring back to FIG. 7 again, the 8×8 motion vectors suitablefor the image size outputted from the image size adjustment flag basedmotion vector adjuster 9 are inputted to the MPEG2 intra-macro blockmotion vector modifier 11. The 8×8 motion vector of each block convertedfrom a macro block which has been an intra-macro block in the bit streamrepresentative of image compression information of MPEG2 is modified bybeing replaced with the 16×16 motion vector determined by the MPEG4 8×8motion vector to MPEG4 16×16 motion vector conversion apparatus 10. The8×8 motion vectors after the modification and the 16×16 motion vectordetermined by the MPEG4 8×8 motion vector to MPEG4 16×16 motion vectorconversion apparatus 10 are outputted as MPEG4 motion vectors.

[0054] A principle of operation of the motion vector modification isdescribed with reference to FIG. 15 which shows a detailed configurationof the MPEG2 intra-macro block motion vector modifier 11 of FIG. 7. An8×8 motion vector suitable for an image size is inputted to the MPEG2intra-macro block motion vector modifier 11. When the intra-mode flag isin a set state, that is, when the macro block is an intra-macro block inthe bit stream which represents image compression information of MPEG2,a movable contact m of a switch SW is connected to a fixed contact aside, and the motion vector of the 8×8 block converted from theintra-macro block is replaced by a replacement apparatus PK with the16×16 motion vector determined by the MPEG4 8×8 motion vector to MPEG416×16 motion vector conversion apparatus 10. In this instance, themotion vector of the 8×8 block may alternatively be replaced with amotion vector converted from a motion vector of an inter-macro blockpresent around the intra-macro block, or otherwise may be replaced witha motion vector converted from a motion vector of an inter-macro blockwhich is nearest to the intra-macro block. It is to be noted that, ifall of the four blocks are converted from intra-macro blocks, the fourmotion vectors become 0, and also the 16×16 motion vector determined bythe MPEG4 8×8 motion vector to MPEG4 16×16 motion vector conversionapparatus 10 becomes 0. Therefore, the motion vector to be used forimage coding of MPEG4 becomes 0, and the macro-block type becomes theintra-mode type. On the other hand, when the intra-mode flag is not in aset state, that is, when no intra-macro block of MPEG2 is involved, themovable contact m of the switch SW is connected to another fixed contactb side, and the 8×8 motion vector inputted is outputted as it is.

[0055] Now, a re-search of a produced motion vector is described. Aconcept of motion vector conversion of an inter-macro block where animage has a frame structure and field prediction is used is illustratedin FIGS. 17A and 17B. FIG. 17A shows a motion vector before resolutionconversion while FIG. 17B shows the motion vector after the resolutionconversion. As described hereinabove with reference to FIGS. 3A and 3B,the horizontal component of a motion vector after conversion can bedetermined from the horizontal component of the motion vector beforeconversion and a resolution conversion rate in the horizontal directionof the image. The vertical component can be determined from the verticalcomponent of the motion vector before the conversion and the resolutionconversion rate in the vertical direction of the image. In other words,if the resolution in the horizontal direction is converted into ½, thenalso the vertical component of the motion vector after the conversionbecomes ½ that before the conversion. The motion vector illustrated inFIGS. 9A and 9B changes, for example, from (8, 12) before conversion to(4, 6) after the conversion. It is to be noted that, in this instance,the distance between middle values (half pixels) of integer pixels isrepresented by 1. Before the resolution conversion of FIG. 17A, eachdark circle indicates the position of an integer pixel, and each rhombindicates the position of a half pixel.

[0056] As can be seen from FIGS. 17A and 17B, a motion vector indicatedat the position of an integer pixel before the conversion is indicatedat the position of an integer pixel or a half pixel after theconversion, but a motion vector indicated at the position of a halfpixel before the conversion does not have a pixel to be referred toafter the resolution conversion. Therefore, where a motion vector beforeconversion indicates the position of a half pixel, the motion vectorafter the conversion is modified so that also it indicates the positionof the nearest integer pixel of a predictive image. This is intended tomake the center pixel of a search window coincide with an integer pixelwhen pixels around the motion vector are re-searched in later motionvector modification. First, it is determined that one of motion vectorsof integer pixels in a search window which exhibits the smallestprediction error, and then half pixel values around the integer pixelare searched thereby to reduce the number of processing steps. Acorrespondence relationship between motion vectors before and afterconversion is illustrated in FIGS. 19A and 19B.

[0057] Alternatively, it is possible to produce a half pixel motionvector and perform a re-search around the half pixel as seen in FIGS.9A, 9B, 10, 11A, 11B, 12A and 12B. In this instance, however, a step ofdetermining the half pixel value in the search window in advance isrequired, resulting in increase of the number of processing steps.

[0058] Now, a concept of motion vector conversion of an inter-macroblock where an image has a frame structure and first field prediction isused as the field prediction is illustrated in FIGS. 19A and 19B. FIG.19A illustrates a motion vector before resolution conversion while FIG.19B illustrates the motion vector after the resolution conversion. Forthe horizontal component of a motion vector, a process similar to thatdescribed hereinabove with reference to FIG. 18 is performed. For thevertical component of the motion vector, the first fields are extractedto allow conversion of the resolution into ½. Further, since first fieldprediction is performed as the prediction, a motion vector before theconversion becomes a motion vector after the conversion as it is.

[0059] A concept of motion vector conversion of an inter-macro blockwhere second field prediction is used is illustrated in FIGS. 20A and20B. FIG. 20A illustrates a motion vector before resolution conversionwhile FIG. 20B illustrates the motion vector after the resolutionconversion. Since only the first fields are extracted upon resolutionconversion, after the conversion, pixel values of the first fields areused as reference images. Field line modification and motion vectormodification in the temporal direction are performed by a method similarto that described hereinabove with reference to FIGS. 12A and 12B, andthereafter, processing similar to that of FIG. 18 is performed for thepixel values in the horizontal direction as seen from integer pixelsafter the conversion.

[0060] Now, a motion vector modification process by a research isdescribed with reference to FIG. 21. In particular, the motion vectormodification apparatus receives a motion vector scaled by the motionvector conversion apparatus 6 of FIG. 2 and outputs a motion vectormodified through a re-search in order to raise the cording efficiency.Further, the motion vector modification apparatus sets a size of asearch window in accordance with a motion vector modification directioninformation flag. The motion vector modification direction informationflag is hereinafter described with reference to FIGS. 23, 24A and 24B.

[0061] In particular, the motion vector modification apparatus performsa re-search to modify a distortion caused by scaling of a motion vectorby the motion vector conversion apparatus 6 in order to determine amotion vector with a higher degree of accuracy. First, the motion vectormodification apparatus performs a motion vector search of ±2 integerpixels in the horizontal direction and ±1 pixel in the verticaldirection around an 8×8 motion vector of MPEG4 inputted thereto.Consequently, the search window centered at the motion vector determinedby the motion vector conversion apparatus 6 can be suppressedhorizontally to 5 pixels and vertically to 3 pixels thereby to reducethe processing number of the motion vector search significantly.Although the search window is set horizontally to 5 pixels andvertically to 3 pixels as just described, the search pixel numbers ofthe window in the horizontal and vertical directions need not be limitedto them and can be selected arbitrarily. In order to reduce the searchprocessing number, a motion vector modification direction informationflag hereinafter described may be used to set the size of the searchwindow for modification of a motion vector asymmetrically in the forwardand reverse directions.

[0062]FIG. 22 shows a configuration of the motion vector conversionapparatus (related art apparatus which is not known when the presentapplication is filed in Japan). The motion vector conversion apparatusis similar to the motion vector conversion apparatus 6 shown in FIG. 2.First, a conversion apparatus 221 performs spatial and temporalmodification of a motion vector described hereinabove to produce an 8×8motion vector. Then, a search apparatus 222 performs a re-search processof an 8×8 motion vector with regard to the 8×8 motion vector asdescribed above. For example, the search apparatus 222 performs asearch, for example, with a search window with two integer pixels in thehorizontal and vertical directions around the reference destination ofthe produced 8×8 motion vector. Consequently, the prediction accuracy ofthe 8×8 motion vector can be improved. Then, prediction errorsre-searched individually with regard to the four 8×8 motion vectorswhich form one macro block and the 8×8 motion vectors for which there-search process has been performed are inputted to a conversionapparatus 223.

[0063] The conversion apparatus 223 discriminates that one of the four8×8 motion vectors which exhibits the smallest prediction error amongthe prediction errors determined upon the re-search for an 8×8 motionvector, and allocates the discriminated motion vector to a 16×16 motionvector. Then, a search apparatus 224 performs a re-search process forthe produced 16×16 motion vector in a similar manner as described aboveto improve the prediction accuracy of the 16×16 motion vector. Thus, the8×8 motion vectors and the 16×16 motion vector of MPEG4 are produced andoutputted.

[0064] The 8×8 motion vectors and the 16×16 motion vector outputted areinputted to a half pixel search apparatus 225, from which 8×8 and 16×16motion vectors of the half pixel accuracy are outputted.

[0065] Here, the motion vector modification direction information flagmentioned hereinabove with reference to FIGS. 21 and 22 is describedwith reference to FIGS. 23A, 23B and 24. The motion vector modificationapparatus sets a size of a re-search window in the advancing directionof a motion vector with the motion vector modification directioninformation flag. First, if the MPEG2 motion vector before conversion isan integer pixel and also the MPEG4 motion vector after conversion hasan integer pixel value as seen in FIG. 23A, since a pixel of the samephase is present and no distortion occurs with the pixel value of themotion vector, a search window for motion vector modification is setsymmetrically in the advancing direction of the motion vector. However,if the MPEG2 motion vector before conversion has an integer pixel valueand the MPEG4 motion vector after the conversion has a half pixelaccuracy as seen in FIG. 23B, in order to modify it to an approximateinteger pixel value of MPEG4 by carrying up in the advancing directionof the motion vector, the search window is set so as to have a greaterpart in the reverse direction to the advancing direction of the motionvector thereby to reduce the search processing number. For example, asearch is performed with two integer pixel values in the reversedirection to the motion vector and with one integer pixel value in theforward direction. Naturally, if the motion vector of MPEG4 is set to anappropriate integer pixel value of MPEG4 by carrying down upon motionvector conversion, then the search window is set so as to be greater inthe forward direction with respect to the advancing direction of themotion vector.

[0066] Also in FIGS. 24A and 24B, the MPEG2 motion vector beforeconversion exhibits a half pixel accuracy. Upon conversion into an MPEGmotion vector, in order to modify the motion vector to the nearestinteger pixel value of MPEG4, the search window of the MPEG4 motionvector modification apparatus can be set, based on the direction inwhich a distortion occurs, so as to be greater in the forward directionor the reverse direction with respect to the advancing direction of themotion vector similarly as in the case of FIG. 23B. Consequently, inorder to reduce the processing number for motion vector re-search, thedirection of the distortion of the motion vector is outputted from themotion vector conversion apparatus 6 and inputted to the motion vectormodification apparatus, by which a size of a search window optimum toperform a modification search for a motion vector can be set.

[0067] The MPEG4 image information coding apparatus 7 receives an outputimage from the resolution frame rate conversion apparatus 5, performscoding of the output image in accordance with an image coding system ofMPEG4 using the motion vector of MPEG4 outputted from the motion vectorconversion apparatus 6, and outputs a bit stream representative of imagecompression information of MPEG4.

[0068] In the MPEG2 motion vector to MPEG4 motion vector conversiondescribed above, since only I and P frames are converted, the frame rateof the bit stream of MPEG4 is 10 frames per second. However, in order toproduce a low bit stream of MPEG4, the frame rate of 10 frames persecond provides a limitation to the cording efficiency and has a problemthat the picture quality is deteriorated significantly.

SUMMARY OF THE INVENTION

[0069] It is an object of the present invention to provide a motionvector conversion method for an image information conversion methodwherein a bit stream representative of interlaced scanned imagecompression information of MPEG2 is inputted and another bit streamrepresentative of progressively scanned image compression information ofMPEG4 is outputted, by which the cording efficiency in image coding ofMPEG4 in the image information conversion method can be augmented.

[0070] It is another object of the present invention to provide a motionvector conversion apparatus for an image information conversionapparatus wherein a bit stream representative of interlaced scannedimage compression information of MPEG2 is inputted and another bitstream representative of progressively scanned image compressioninformation of MPEG4 is outputted, by which the cording efficiency inimage coding of MPEG4 in the image information conversion method can beaugmented.

[0071] In order to attain the objects described above, according to anaspect of the present invention, there is provided a motion vectorconversion method for an image information conversion method wherein abit stream representative of interlaced scanned image compressioninformation of MPEG2 is inputted and a bit stream representative ofprogressively scanned image compression information of MPEG4 isoutputted, comprising the steps of successively accepting 16×16 motionvectors of MPEG2 of the inputted bit stream representative of imagecompression information of MPEG2, and successively producing 8×8 motionvectors of MPEG4 and 16×16 motion vectors of MPEG4 based on the 16×16motion vectors of MPEG2 such that every other one of I frames and Pframes of the bit stream of MPEG2 is dropped to produce a bit stream ofMPEG4 of a reduced frame rate and a low bit rate.

[0072] In the motion vector conversion method for an image informationconversion method wherein a bit stream representative of interlacedscanned image compression information of MPEG2 is inputted and a bitstream representative of progressively scanned image compressioninformation of MPEG4 is outputted, 16×16 motion vectors of MPEG2 of theinputted bit stream representative of image compression information ofMPEG2 are accepted successively, and 8×8 motion vectors of MPEG4 and16×16 motion vectors of MPEG4 are successively produced based on the16×16 motion vectors of MPEG2 such that every other one of I frames andP frames of the bit stream of MPEG2 is dropped to produce a bit streamof MPEG4 of a reduced frame rate and a low bit rate. Consequently, themotion vector conversion method can achieve a high coding efficiency inimage coding of MPEG4 in the image information conversion method.

[0073] According another aspect of the present invention, there isprovided a motion vector conversion apparatus for an image informationconversion apparatus to which a bit stream representative of interlacedscanned image compression information of MPEG2 is inputted and fromwhich a bit stream representative of progressively scanned imagecompression information of MPEG4 is outputted, comprising motion vectorproduction means for accepting 16×16 motion vectors of MPEG2 of theinputted bit stream representative of image compression information ofMPEG2 and successively producing 8×8 motion vectors of MPEG4 and 16×16motion vectors of MPEG4, and dropping means for dropping every other oneof I frames and P frames of the inputted bit stream of MPEG2 andsupplying 16×16 motion vectors of MPEG2 of the remaining I frames and Pframes to the motion vector production means so that a bit stream ofMPEG4 of a reduced frame rate and a low bit rate may be produced by themotion vector production means.

[0074] In the motion vector conversion apparatus for an imageinformation conversion apparatus to which a bit stream representative ofinterlaced scanned image compression information of MPEG2 is inputtedand from which a bit stream representative of progressively scannedimage compression information of MPEG4 is outputted, the motion vectorproduction means accepts 16×16 motion vectors of MPEG2 of the inputtedbit stream representative of image compression information of MPEG2 andsuccessively produces 8×8 motion vectors of MPEG4 and 16×16 motionvectors of MPEG4. The dropping means drops every other one of I framesand P frames of the inputted bit stream of MPEG2 and supplies 16×16motion vectors of MPEG2 of the remaining I frames and P frames to themotion vector production means. Therefore, the motion vector productionmeans produces a bit stream of MPEG4 of a reduced frame rate and a lowbit rate. Consequently, the motion vector conversion apparatus canachieve a high coding efficiency in image coding of MPEG4 in an imageinformation conversion method.

[0075] The above and other objects, features and advantages of thepresent invention will become apparent from the following descriptionand the appended claims, taken in conjunction with the accompanyingdrawings in which like parts or elements denoted by like referencesymbols.

BRIEF DESCRIPTION OF THE DRAWINGS

[0076]FIG. 1 is a block diagram showing a related art image informationconversion apparatus which converts a bit stream representative of imagecompression information of MPEG2 into another bit stream representativeof image compression information of MPEG4;

[0077]FIG. 2 is a block diagram showing an image information conversionapparatus (related art) proposed by the inventors of the presentinvention which converts a bit stream representative of imagecompression information of MPEG2 into another bit stream representativeof image compression information of MPEG4;

[0078]FIGS. 3A and 3B are diagrammatic views illustrating a correlationbetween a motion vector in a bit stream representative of imagecompression information of MPEG2 and another motion vector in anotherbit stream representative of image compression information of MPEG4;

[0079]FIG. 4 is a diagrammatic view illustrating a principle ofoperation of a resolution frame rate conversion apparatus in the imageinformation conversion apparatus of FIG. 2;

[0080]FIG. 5 is a schematic view illustrating a principle of operationof supplementing or removing a pixel of the resolution frame rateconversion apparatus in the image information conversion apparatus ofFIG. 2 in accordance with an image frame size adjustment flag;

[0081]FIGS. 6A and 6B are diagrammatic views illustrating a motionvector conversion method of a motion vector conversion apparatus in theimage information conversion apparatus of FIG. 2;

[0082]FIG. 7 is a block diagram showing a detailed configuration of themotion vector conversion apparatus in the image information conversionapparatus of FIG. 2;

[0083]FIG. 8 is a flow chart illustrating a principle of operation of anMPEG2 16×16 motion vector to MPEG4 8×8 motion vector conversionapparatus in the image information conversion apparatus of FIG. 2;

[0084]FIGS. 9A and 9B are diagrammatic views illustrating a concept ofmotion vector conversion of the MPEG2 16×16 motion vector to MPEG4 8×8motion vector conversion apparatus in the motion vector conversionapparatus of FIG. 7 wherein a frame structure and frame prediction areused;

[0085]FIG. 10 is a table illustrating handling after conversion into anMPEG4 8×8 motion vector of a motion vector of a half pixel accuracy in abit stream representative of image compression information of MPEG2illustrated in FIG. 9;

[0086]FIGS. 11A and 11B are diagrammatic views illustrating motionvector conversion wherein the image illustrated in FIG. 9 has a framestructure and first field prediction is used;

[0087]FIGS. 12A and 12B are diagrammatic views illustrating motionvector conversion wherein the image illustrated in FIG. 9 has a framestructure and second field prediction is used;

[0088]FIG. 13 is a diagrammatic view illustrating motion vectorconversion wherein the image illustrated in FIG. 9 has a frame structureand second field prediction is used;

[0089]FIG. 14 is a flow chart illustrating operation of a motion vectormodifier which operates with an image size adjustment flag in the motionvector conversion apparatus of FIG. 7;

[0090]FIG. 15 is a block diagram illustrating a principle of operationof the motion vector modifier shown in FIG. 7 which performs motionvector modification for an intra-macro block of MPEG2;

[0091]FIG. 16 is a block diagram showing a detailed configuration of themotion vector conversion apparatus in the image information conversionapparatus of FIG. 2;

[0092]FIGS. 17A and 17B are diagrammatic views illustrating a concept ofmotion vector conversion by the MPEG2 16×16 motion vector to MPEG4 8×8motion vector conversion apparatus, which performs a motion vectorre-search, in the motion vector conversion apparatus of FIG. 7 wherein aframe structure and frame prediction are used;

[0093]FIG. 18 is a table illustrating handling after conversion into anMPEG4 8×8 motion vector of a motion vector of a half pixel accuracy in abit stream representative of image compression information of MPEG2 whena motion vector re-search is performed in the motion vector conversionapparatus of FIG. 7;

[0094]FIGS. 19A and 19B are diagrammatic views illustrating a concept ofmotion vector conversion wherein the image when a motion-vectorre-search is performed by the motion vector conversion apparatus of FIG.7 has a frame structure and first field prediction is used;

[0095]FIGS. 20A and 20B are diagrammatic views illustrating a concept ofmotion vector conversion wherein the image when a motion-vectorre-search is performed by the motion vector conversion apparatus of FIG.7 has a frame structure and second field prediction is used;

[0096]FIG. 21 is a block diagram showing a motion vector modificationapparatus;

[0097]FIG. 22 is a block diagram showing a motion vector re-searchapparatus which performs a re-search process for an 8×8 motion vectorand produces a 16×16 motion vector based on a predictive residual;

[0098]FIGS. 23A and 23B are diagrammatic views illustrating a distortionof a motion vector by spatial modification;

[0099]FIGS. 24A and 24B are diagrammatic views illustrating a distortionof a motion vector by spatial modification;

[0100]FIG. 25 is a diagrammatic view illustrating dropping of an I frameaccording to an embodiment of the present invention;

[0101]FIGS. 26, 27 and 28 are diagrammatic views illustrating droppingof a P frame according to the embodiment of the present invention;

[0102]FIG. 29 is a diagrammatic view illustrating in what manner areference macro block overlaps with a plurality of macro blocksaccording to the embodiment of the present invention;

[0103]FIG. 30 is a flow chart illustrating a motion vector synthesisalgorithm where a P frame is dropped according to the embodiment of thepresent invention;

[0104]FIG. 31 is a diagrammatic view illustrating temporal modificationof a motion vector wherein the second field of a motion vector isreferred to according to the embodiment of the present invention;

[0105]FIG. 32 is a block diagram showing an example of a motion vectorconversion apparatus according to the embodiment of the presentinvention;

[0106]FIG. 33 is a block diagram showing another example of the motionvector conversion apparatus according to the embodiment of the presentinvention;

[0107]FIG. 34 is a block diagram showing a further example of the motionvector conversion apparatus according to the embodiment of the presentinvention; and

[0108]FIG. 35 is a block diagram showing a still further example of themotion vector conversion apparatus according to the embodiment of thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0109]FIG. 25 illustrates a concept of a correspondence relationshipbetween a picture type and a VOP (Video Object Plane) type uponconversion from an MPEG2 stream into an MPEG4 stream. When a stream ofMPEG2 of 30 frames per second wherein the GOP (Group Of Pictures)structure is N15M3 is to be converted into another stream of MPEG4 of 5frames per second wherein the GOP structure is N5M1, every other ones ofI and P frames are converted into MPEG4 frames. Since one GOP of N5M1 ofthe stream of MPEG4 corresponds to two GOPs of N15M1 of the stream ofMPEG2 as seen in FIG. 25, upon conversion from MPEG2 into MPEG4, twocases are available including a case wherein a P frame is dropped andanother case wherein an I frame is dropped.

[0110] First, dropping of an I frame is described with reference to FIG.26. Since an I frame I_(n−1) between P frames P_(n−2) and P_(n) does notinclude any motion vector, the motion vector of the dropped I framecannot be added to the motion vector MV_(n) of the pertaining macroblock to synthesize a motion vector. Therefore, in the pertaining macroblock, the motion vector MV_(n) of the macro block is spatially andtemporally modified by scaling in accordance with the method describedhereinabove in the description of the related art, and then the lengthof the motion vector MV_(n) in the temporal direction is extended totwice so as to refer to a VOP of MPEG4 converted from an I or P framepreceding to the dropped P frame to obtain a motion vector 2*MV_(n).Consequently, the VOP converted from the I or P frame immediatelypreceding to the frame whose motion vector of the P-VOP is referred to.In this manner, scaling by spatial and temporal modification isperformed, and then, a motion vector extended to twice in the temporaldirection is produced and outputted as an 8×8 motion vector.

[0111] Now, dropping of a P frame is described with reference to FIGS.27, 28, 29 and 30. Referring first to FIG. 27, there is illustrated aconcept of a process when a P frame is dropped. Since a P frame P_(n−1)to be dropped between an I frame I_(n−2) and a P frame P_(n) has amotion vector MV_(n−1), the motion vector MV_(n−1) of the P frame to bedropped is added to the motion vector MV_(n) of the pertaining macroblock to produce a motion vector MV_(n−1)+MV_(n1) which refers to theframe preceding to the P frame to be dropped.

[0112] As shown in FIG. 28, a motion vector of the macro block in the Pframe next to the P frame to be dropped refers to the P frame to bedropped. In this case, the reference area overlaps with a plurality ofmacro blocks to be referred to in the P frame to be dropped.

[0113]FIG. 29 illustrates in what manner the area MB₀ for referenceoverlaps with a plurality of macro blocks. As can be seen from FIG. 29,the area MB₀ for reference may possibly overlap with one, two or fourmacro blocks (MB). The portions at which the area MB₀ for referenceoverlaps with the macro blocks MB are denoted by refMB#0, refMB#1,refMB#2 and refMB#3. Motion vectors, macro block modes and predictionmodes, bit numbers or quantization scales of the P frame to be droppedare stored in a unit of a frame into a motion vector and macro blockinformation buffer in order to allow later synthesis of motion vectorsby addition. With regard to the pertaining macro block for which motionvector conversion from MPEG2 to MPEG4 is to be performed, coordinates tobe referred to by the motion vector on the dropped P frame arecalculated. As seen from FIG. 29, depending upon the coordinatesreferred to by the motion vector, the area MB₀ for reference maypossibly overlap with one, two or four macro blocks (MB) of thereference P frame dropped. Where the area MB₀ for reference overlapswith a plurality of macro blocks, the coding efficiencies of the macroblocks are re-arranged in a descending order based on a parameter Xdetermined in advance.

[0114] In particular, in a first method, the parameter X is the “numberof pixels in each of portions, overlapping with the macro block in the Pframe next to the P frame to be dropped, of macro blocks in the P frameto be dropped”; in a second method, the parameter X is the “number ofpixels in each of portions, overlapping with the macro block in the Pframe next to the P frame to be dropped, of macro blocks in the P frameto be dropped /macro block bit number”; in a third method, the parameterX is the “number of pixels in each of portions, overlapping with themacro block in the P frame next to the P frame to be dropped, of macroblocks in the P frame to be dropped/Q-scale” (the Q scale signifies amacro block quantization scale); in a fourth method, the parameter X isthe “number of pixels in each of portions, overlapping with the macroblock in the P frame next to the P frame to be dropped, of macro blocksin the P frame to be dropped/(macro block bit number×Q-scale”; in afifth method, the parameter X is 1/macro block bit number; in a sixthmethod, the parameter X is the (1/Q-scale); and in a seventh method, theparameter X is 1/(macro block bit number×Q-scale). The highest one ofthe values of the parameter X is determined as a maximum value, and thelowest one of the values of the parameter X is determined as a minimumvalue. In FIG. 28, the parameters X according to the first to fourthmethods described above are illustrated. In FIG. 28,“MB_(overlapped area)” denotes the number of pixels in each of portions,overlapping with the macro block in the P frame next to the P frame tobe dropped, of macro blocks in the P frame to be dropped”; “Coef bits”denotes the macro bit number; and “Q-scale” denotes the macro blockquantization scale.

[0115] Now, a motion vector synthesis algorithm where a P frame isdropped is described with reference to a flow chart of FIG. 30. First,from among one, two or four macro blocks (MB) in the P frame to bedropped, which overlap with an MPEG2 motion vector of the pertainingarea in order to be referred to, those in which the macro block (MB)mode is Not Coded (whose DCT coefficients are not coded) and a number ofpixels in the overlapping macro block is greater than a threshold valueT are searched for (step ST-21). The threshold value T is set, forexample, to 100 pixels. If at least one macro block (MB) of Not Coded isincluded in the overlapping macro blocks, then the motion vectorMV_(n−1) of the macro block (MB) of Not Coded having the lowest macroblock (MB) address is selected (step ST-22).

[0116] If a macro block of Not Coded (whose DCT coefficients are notcoded) is not included in the one, two or four macro blocks in the Pframe to be dropped, which overlap with the MPEG2 motion vector of thepertaining area in order to be referred to, then macro blocks in whichthe macro block (MB) mode is No MC (no motion compensation) and a numberof pixels in the overlapping macro block is greater than the thresholdvalue T are searched for (step ST-23). The threshold value T is set, forexample, to 100 pixels. If at least one macro block of No MC is includedin the overlapping macro blocks, then the motion vector MV_(n−1) of themacro blocks of No MC having the lowest macro block address is selected(step ST-24).

[0117] If a macro block of Not Coded (whose DCT coefficients are notcoded) or No MC (No motion compensation) is not included in the one, twoor four macro blocks which overlap with the MPEG2 motion vector of thepertaining macro block in order to be referred to, then it isdiscriminated in order beginning with the macro block which has themaximum parameter X described above whether or not the macro block (MB)mode of each of the macro blocks is an intra-macro block (MB (stepST-25) If the reference macro block is an intra-macro block, then it isdiscriminated whether or not the macro block which has the secondmaximum parameter X is an intra-macro block and it is discriminatedwhether or not all of the macro blocks which overlap with the pertainingmacro block are intra-macro blocks in order to be referred to (stepST-26). If all of the macro blocks which overlap with the pertainingmacro block in order to be referred to are intra-macro blocks, then azero motion vector (MV) is selected (step ST-27).

[0118] If a macro block which is not an intra-macro block is searchedout in step ST-25, then the processing advances to a next discriminationroutine. In particular, it is discriminated whether or not theprediction mode of the macro block which is not an intra-macro block isfield prediction wherein the second field is referred to (step ST-28).If the field prediction wherein the second field is referred to is used,then similar discrimination is performed for the reference block whichhas the second maximum parameter X (step ST-29). If N (the number ofintra-macro blocks) reference macro blocks use the field predictionwherein the second field is referred to, then the motion vector MV_(n−1)of the reference macro block (MB) which has the highest parameter X(step ST-30) is selected. N is a number determined in advance and hasthe value of 1, 2, 3 or 4. If a macro block which does not use the fieldprediction wherein the second field is referred to is searched out, thenthe motion vector MV_(n−1) of the macro block is selected (step ST-31).A motion vector to be added to the motion vector of the pertaining macroblock is selected in this manner.

[0119]FIG. 31 illustrates temporal modification of a motion vector whichrefers to the second field. Scaling (resolution conversion) of themotion vector obtained by the addition is performed in accordance with amethod similar to the scaling method wherein spatial and temporalmodification is performed for a motion vector described hereinabove inthe description of the related art. Consequently, if the motion vectorobtained by the addition refers to the second field of the referenceframe, then field modification is performed for the vertical componentof the motion vector, and then the motion vector is extended by anamount equal to one field distance in the temporal direction as vectormodification. As seen in FIG. 31, the field distance between thepertaining field and the second field to be referred to is denoted by a,and therefore, the motion vector is multiplied by (a+1)/a in order totemporally modify the same for one field. An 8×8 motion vector isdetermined in this manner and outputted.

[0120]FIG. 32 shows a configuration of a motion vector conversionapparatus which drops an I frame. Referring to FIG. 32, the motionvector conversion apparatus is similar to the motion vector conversionapparatus 6 described hereinabove with reference to FIG. 2. First,motion vectors of MPEG2 are inputted to a motion vector spatial-temporalmodification apparatus 321, by which spatial and temporal modificationof the motion vectors is performed in a similar manner as describedhereinabove to produce 8×8 motion vectors. Then, the 8×8 motion vectorsare inputted to an 8×8 MV re-search apparatus 322, in which a re-searchprocess for 8×8 motion vectors is performed as described hereinabove inthe description of the related art. For example, a search windowcentered at a reference destination of each of the produced 8×8 motionvectors is produced with two integer pixels in the horizontal andvertical directions, and a search is performed within the search window.Consequently, the prediction accuracy of the 8×8 motion vector can beimproved. Then, prediction errors researched with regard to four 8×8motion vectors which form one macro block and the 8×8 motion vectors forwhich the re-search process has been performed are inputted to an 8×8 MVto 16×16 MV conversion apparatus 323.

[0121] The 8×8 MV to 16×16 MV conversion apparatus 323 discriminatesthat one of the four 8×8 motion vectors which has the lowest predictionerror and allocates the discriminated motion vector to the 16×16 motionvector. Then, the produced 16×16 motion vector is inputted to a 16×16 MVre-search apparatus 324, by which a re-search process is performedsimilarly as described in the description of the related art to improvethe prediction accuracy of the 16×16 motion vector. The 8×8 motionvectors and the 16×16 motion vector of MPEG4 produced in this manner areoutputted.

[0122]FIG. 33 shows a configuration of a motion vector conversionapparatus which drops a P frame. Referring to FIG. 33, the motion vectorconversion apparatus is similar to the motion vector conversionapparatus 6 described hereinabove with reference to FIG. 2. First,motion vectors of MPEG2 are inputted to a changeover switch 331. Thechangeover switch 331 is switched to an MV and MB information buffer 332side when the motion vector of MPEG2 inputted is information of a Pframe to be dropped, but is switched to a reference MB addresscalculation apparatus 334 side when the motion vector of MPEG2 inputtedis information of any other P frame.

[0123] The MV and MB information buffer 332 stores motion vectorinformation and macro block prediction modes of P frames to be droppedin a unit of frames and, according to circumstances, stores the bitamount and the quantization scale of each of the macro blocks. Thereference MB address calculation apparatus 334 accepts motion vectors ofa P frame next to a P frame to be dropped and calculates a referenceposition of each of the motion vectors. The reference position of themotion vector is inputted to an addition mode vector discriminator 333,by which macro blocks of the P frame to be dropped which overlap withthe reference macro block are calculated based on the referenceposition. The addition mode vector discriminator 333 makes use of thisinformation to perform processing based on such a technique of selectinga motion vector to be added as described above, and outputs a motionvector to be added. The outputted motion vector is added to the motionvector of the pertaining macro block to produce a synthesized motionvector.

[0124] Motion vectors synthesized in this manner are inputted to amotion vector spatial-temporal modification apparatus 335, by whichspatial and temporal modification scaling is performed for the motionvectors in a similar manner as described hereinabove in the descriptionof the related art and in the paragraphs given hereinabove to produce8×8 motion vectors. An 8×8 MV re-search apparatus 336, an 8×8 MV to16×16 MV conversion apparatus 337 and a 16×16 motion vector re-searchapparatus 338 operate similarly to the 8×8 MV re-search apparatus 322,8×8 MV to 16×16 MV conversion apparatus 323 and 16×16 MV re-searchapparatus 324 of the motion vector conversion apparatus of FIG. 32,respectively, to produce a 16×16 motion vector. The 8×8 motion vectorsand the 16×16 motion vector of MPEG4 produced in this manner areoutputted. In this manner, an 8×8 motion vector of MPEG4 can bedetermined for each of four blocks which form one macro block by are-search. Prediction errors determined by the re-search of the four 8×8motion vectors of MPEG4 are compared with one another to discriminatethe 8×8 motion vector which has the smallest prediction error. The 8×8motion vector having the smallest prediction error is allocated to the16×16 motion vector.

[0125] Also for the selected 16×16 motion vector, a re-search isperformed in a similar manner as in the 8×8 motion vector re-searchmethod described hereinabove to optimize the coding efficiency of the16×16 motion vector.

[0126] In this manner, in a procedure wherein image informationcompression information of MPEG2 is inputted to determine 8×8 motionvectors and 16×16 motion vectors of MPEG4, the motion vector conversionapparatus extends or adds the motion vectors and performs motion vectormodification by a re-search of the scaled motion vector informationcentered at each of the motion vectors. Consequently, a drop of thecoding efficiency of an image coding apparatus of MPEG4 can beminimized.

[0127]FIG. 34 shows another configuration of the motion vectorconversion apparatus. It is to be noted that, before description withreference to FIG. 34, similar description to that given hereinabove withreference to FIGS. 25 and 26 must be given. For the description,however, the description given hereinabove with reference to FIGS. 25and 26 are quoted. Referring to FIG. 34, the motion vector conversionapparatus shown is similar to the motion vector conversion apparatus 6described hereinabove with reference to FIG. 2. First, an MPEG4 to MPEG28×8 MV conversion apparatus 271 performs spatial and temporalmodification of motion vectors described above to produce 8×8 motionvectors. Then, a motion vector integer pixel search apparatus 272performs a re-search process for each of the 8×8 motion vectors asdescribed hereinabove in the description of the related art. Forexample, a search window centered at a reference destination of each ofthe produced 8×8 motion vectors is produced with two integer pixels inthe horizontal and vertical directions, and a search is performed withinthe search window. Consequently, the prediction accuracy of the 8×8motion vector can be improved. Then, prediction errors re-searched withregard to four 8×8 motion vectors which form one macro block and the 8×8motion vectors for which the re-search process has been performed areinputted to an 8×8 motion vector to 16×16 motion vector conversionapparatus 273.

[0128] The 8×8 motion vector to 16×16 motion vector conversion apparatus273 discriminates that one of the four 8×8 motion vectors which has thelowest prediction error among the prediction errors determined upon the8×8 motion vector re-search and allocates the discriminated motionvector to the 16×16 motion vector. Then, the produced 16×16 motionvector is inputted to a motion vector integer pixel search apparatus274, by which a re-search process is performed similarly as describedhereinabove in the description of the related art to improve theprediction accuracy of the 16×16 motion vector. The 8×8 motion vectorsand the 16×16 motion vector of MPEG4 produced in this manner areoutputted.

[0129] The 8×8 motion vectors and the 16×16 motion vector of MPEG4outputted are inputted to a half pixel search apparatus 275, from whichthe 8×8 and 16×16 motion vectors are outputted with a half pixelaccuracy.

[0130] In this manner, in a procedure wherein image informationcompression information of MPEG2 is inputted to determine 8×8 motionvectors and 16×16 motion vectors of MPEG4, the motion vector conversionapparatus extends or adds the motion vectors and performs motion vectormodification by a re-search of the scaled motion vector informationcentered at each of the motion vectors. Consequently, a drop of thecoding efficiency of an image coding apparatus of MPEG4 can beminimized.

[0131]FIG. 35 shows a further configuration of the motion vectorconversion apparatus wherein a P frame is dropped. It is to be notedthat, before description with reference to FIG. 35, similar descriptionto that given hereinabove with reference to FIGS. 25 to 32 must begiven. For the description, however, the description given hereinabovewith reference to FIGS. 25 to 32 are quoted. Referring to FIG. 35, themotion vector conversion apparatus shown is similar to the motion vectorconversion apparatus 6 described hereinabove with reference to FIG. 2.First, motion vectors of MPEG2 are inputted to a changeover switch 351.The changeover switch 351 is switched to an MV and MB information buffer352 side when information of a P frame to be dropped is received, but isswitched to a reference MB address calculation apparatus 354 and amotion vector extension spatial-temporal modification apparatus 357 sidewhen information of any other P frame is received.

[0132] The MV and MB information buffer 352 stores motion vectorinformation and macro block prediction modes of P frames to be droppedin a unit of a frame and, according to circumstances, stores the bitamount and the quantization scale of each of the macro blocks. Thereference MB address calculation apparatus 354 accepts motion vectors ofa P frame next to a P frame to be dropped and calculates a referenceposition of each of the motion vectors. The reference position of themotion vector is inputted to an addition motion vector discriminationapparatus 353, by which macro blocks of the P frame to be dropped whichoverlap with the reference macro block are calculated based on thereference position. The addition motion vector discrimination apparatus353 makes use of this information to perform processing based on such atechnique of selecting a motion vector to be added as described above,and outputs a motion vector to be added. The outputted motion vector isadded to the motion vector of the pertaining macro block to produce asynthesized motion vector.

[0133] Motion vectors synthesized in this manner are inputted to amotion vector spatial-temporal modification apparatus 355, by whichspatial and temporal modification scaling is performed for the motionvectors in a similar manner as described hereinabove in the descriptionof the related art and in the paragraphs given hereinabove to produce8×8 motion vectors. the motion vector extension spatial-temporalmodification apparatus 357 and a motion vector re-search apparatus 358are used to determine a motion vector in accordance with the extensionmethod and perform processes similar to those of the motion vectorspatial-temporal modification apparatus 321 and the 8×8 motion vectorre-search apparatus 322, respectively. Consequently, motion vectors areproduced by them. The motion vectors outputted from the motion vectorre-search apparatus 356 and the motion vector re-search apparatus 358are inputted to an 8×8 motion vector selection apparatus 359. The 8×8motion vector selection apparatus 359 outputs those of the motionvectors from the motion vector re-search apparatus 356 and the motionvector re-search apparatus 358 which have smaller predictive residualsas 8×8 motion vectors. An 8×8 MV to 16×16 MV conversion apparatus 60 anda 16×16 motion vector re-search apparatus 361 operate similarly to the8×8 MV to 16×16 MV conversion apparatus 323 and 16×16 MV re-searchapparatus 324 of the motion vector conversion apparatus of FIG. 32,respectively, to produce a 16×16 motion vector. The 8×8 motion vectorsand the 16×16 motion vector of MPEG4 produced in this manner areoutputted.

[0134] In this manner, when a P frame is dropped, two different kinds ofmotion vectors including motion vectors determined by extension ofmotion vectors of four blocks which form one macro block and a motionvector determined by addition of the motion vectors. A re-search processis performed subsequently for the motion vectors and a motion vectorwhich exhibits the smallest prediction error is outputted as an 8×8motion vector of MPEG4. The method of the re-search may be similar tothe method described hereinabove in the description of the related art,and, for example, the range of search of the search window is twointeger pixels in the vertical and horizontal directions. Consequently,two motion vectors can be determined by a re-search for each of fourblocks which form one macro block.

[0135] Then, prediction errors of the two motion vectors for one blockare compared with each other, and one of the motion vectors whichexhibits the lower predictive residual is outputted as an 8×8 motionvector. In this manner, between motion vectors determined by the methodwherein a motion vector is extended and the method wherein motionvectors are added, the motion vector which has the highest codingefficiency can be selected to produce an 8×8 motion vector. Thereafter,prediction errors determined by a re-search of the four 8×8 motionvectors of MPEG4 of a macro bloc are compared with one another todiscriminate the 8×8 motion vector which exhibits the smallestprediction error. The 8×8 motion vector which exhibits the smallestprediction error is allocated to a 16×16 motion vector. Then, are-search is performed also for the selected 16×16 motion vectorsimilarly as in the 8×8 motion vector re-search method describedhereinabove to optimize the coding efficiency of the 16×16 motionvector.

[0136] In this manner, in a procedure wherein image informationcompression information of MPEG2 is inputted to determine 8×8 and 16×16motion vectors of MPEG4, the motion vector conversion apparatus extendsor adds the motion vectors and performs motion vector modification by are-search of the scaled motion vector information centered at each ofthe motion vectors. Consequently, a drop of the coding efficiency of animage coding apparatus of MPEG4 can be minimized.

[0137] Although a bit stream representative of image compressioninformation of MPEG2 is inputted and a bit stream representative ofimage compression information of MPEG4 is outputted as described above,the input and the output are not limited to the specific ones described,but they may otherwise be bit streams representative of imagecompression information, for example, of MPEG-1 or H.263.

[0138] While preferred embodiments of the present invention have beendescribed using specific terms, such description is for illustrativepurposes only, and it is to be understood that changes and variationsmay be made without departing from the spirit or scope of the followingclaims.

What is claimed is:
 1. A motion vector conversion method for an imageinformation conversion method wherein a bit stream representative ofinterlaced scanned image compression information of MPEG2 is inputtedand a bit stream representative of progressively scanned imagecompression information of MPEG4 is outputted, comprising the steps of:successively accepting 16×16 motion vectors of MPEG2 of the inputted bitstream representative of image compression information of MPEG2; andsuccessively producing 8×8 motion vectors of MPEG4 and 16×16 motionvectors of MPEG4 based on the 16×16 motion vectors of MPEG2 such thatevery other one of I frames and P frames of the bit stream of MPEG2 isdropped to produce a bit stream of MPEG4 of a reduced frame rate and alow bit rate.
 2. A motion vector conversion method according to claim 1,wherein said motion vector conversion method serves as motion vectormodification; and motion vector information and macro block informationin the inputted bit stream representative of image compressioninformation of MPEG2 in each P frame to be dropped are stored into abuffer, and a motion vector to be added in said frame to be dropped isdetermined based on the information stored in said buffer, whereafterthe determined motion vector is added, to produce an 8×8 motion vector.3. A motion vector conversion method according to claim 2, whereinletting a parameter X be taken as a parameter associated with eachmotion vector in said P frame to be dropped, a motion vector having thehighest parameter X in said P frame to be dropped is determined as saidmotion vector to be added in said P frame to be dropped.
 4. A motionvector conversion method according to claim 3, wherein the parameter Xis a parameter selected from the group consisting of the number ofpixels in each of portions of one or more macro blocks in said P frameto be dropped, which portions overlap with a macro block of a motionvector of the P frame next to said P frame to be dropped in order to bereferred to; a value obtained by dividing the number of pixels in eachof portions of one or more macro blocks in said P frame to be dropped,which portions overlap with a macro block of a motion vector in the Pframe next to said P frame to be dropped in order to be referred to, bya macro block bit number; a value obtained by dividing the number ofpixels in each of portions of one or more macro blocks in said P frameto be dropped, which portions overlap with a macro block of a motionvector in the P frame next to said P frame to be dropped in order to bereferred to, by a macro block quantization scale; and a value obtainedby dividing the number of pixels in each of portions of one or moremacro blocks in said P frame to be dropped, which portions overlap witha macro block of a motion vector in the P frame next to said P frame tobe dropped in order to be referred to, by the product of a macro blockquantization scale and the macro block bit number.
 5. A motion vectorconversion method according to claim 2, wherein letting a parameter X betaken as a parameter associated with each motion vector in said P frameto be dropped, a motion vector having the lowest parameter X in said Pframe to be dropped is determined as said motion vector to be added insaid P frame to be dropped.
 6. A motion vector conversion methodaccording to claim 5, wherein the parameter X is a parameter selectedfrom the group consisting of a macro block bit number in each ofportions of one or more macro blocks in said P frame to be dropped,which portions overlap with a macro block of a motion vector of the Pframe next to said P frame to be dropped in order to be referred to; amacro block quantization scale in each of portions of one or more macroblocks in said P frame to be dropped, which portions overlap with amacro block of a motion vector in the P frame next to said P frame to bedropped in order to be referred to; and the product of a macro blockquantization scale and the macro block bit number in each of portions ofone or more macro blocks in said P frame to be dropped, which portionsoverlap with a macro block of a motion vector of the P frame next tosaid P frame to be dropped in order to be referred to.
 7. A motionvector conversion method according to claim 2, wherein in the case whereany of one or more macro blocks in said frame to be dropped, whichoverlap with a macro block of a motion vector in the P frame next tosaid P frame to be dropped in order to be referred to, is different froma macro block whose DCT coefficients are not coded and a macro blockfree from motion compensation, if all of the overlapping macro blocks insaid frame to be dropped are intra-macro blocks, then a zero motionvector is selected.
 8. A motion vector conversion method according toclaim 2, wherein in the case where any of one or more macro blocks insaid frame to be dropped, which overlap with a macro block of a motionvector in the P frame next to said P frame to be dropped in order to bereferred to, is different from a macro block whose DCT coefficients arenot coded and a macro block free from motion compensation and furtherthe overlapping macro blocks in said frame to be dropped contains atleast one macro block whose macro block mode is not an intra-macroblock, if a prediction mode carried out for those of the overlappingmacro blocks, whose macro-block mode is not an intra-macro block, is asecond field prediction, then a motion vector of a macro block havingthe highest parameter X is selected.
 9. A motion vector conversionmethod according to claim 8, wherein the parameter X is a parameterselected from the group consisting of the number of pixels in each ofportions of one or more macro blocks in said P frame to be dropped,which portions overlap with a macro block of a motion vector in the Pframe next to said P frame to be dropped in order to be referred; avalue obtained by dividing the number of pixels in each of portions ofone or more macro blocks in said P frame to be dropped, which portionsoverlap with a macro block of a motion vector in the P frame next tosaid P frame to be dropped in order to be referred to, by a macro blockbit number; a value obtained by dividing the number of pixels in each ofportions of one or more macro blocks in said P frame to be dropped,which portions overlap with a macro block of a motion vector in the Pframe next to said P frame to be dropped in order to be referred to, bya macro block quantization scale; and a value obtained by dividing thenumber of pixels in each of portions of one or more macro blocks in saidP frame to be dropped, which portions overlap with a macro block of amotion vector in the P frame next to said P frame to be dropped in orderto be referred to, by the product of a macro block quantization scaleand the macro block bit number.
 10. A motion vector conversion methodaccording to claim 1, wherein said motion vector conversion methodserves as motion vector modification; and a motion vector in the P framenext to said I frame to be dropped is extended twice in the temporaldirection based on motion vector information and macro block informationin the inputted bit stream representative of image compressioninformation of MPEG2, to produce an 8×8 motion vector.
 11. A motionvector conversion method according to claim 1, wherein said motionvector conversion method serves as motion vector modification; and an8×8 motion vector of MPEG4 is obtained from a motion vector of MPEG2 bya motion vector conversion method based on motion vector information inthe inputted bit stream representative of image compression informationof MPEG2 and is inputted; and a re-search for a motion vector centeredat the inputted 8×8 motion vector value is performed to modify themotion vector such that one of four 8×8 motion vectors of MPEG4cooperatively forming one macro block, which has the smallest predictionresidual, is allocated to a 16×16 motion vector, to produce the 16×16motion vector.
 12. A motion vector conversion method according to claim1, wherein in the case where any of one or more macro blocks in saidframe to be dropped, which overlap with a macro block of a motion vectorin the P frame next to said P frame to be dropped in order to bereferred to, is different from a macro block whose DCT coefficients arenot coded and a macro block free from motion compensation, if all of theoverlapping macro blocks in said frame to be dropped are intra-macroblocks, then a zero motion vector is selected.
 13. A motion vectorconversion method according to claim 1, wherein in the case where any ofone or more macro blocks in said frame to be dropped, which overlap witha macro block of a motion vector in the P frame next to said P frame tobe dropped in order to be referred to, is different from a macro blockwhose DCT coefficients are not coded and a macro block free from motioncompensation and further the overlapping macro blocks in said frame tobe dropped contains at least one macro block whose macro block mode isnot an intra-macro block, if a prediction mode carried out for those ofthe overlapping macro blocks, whose macro-block mode is not anintra-macro block, is a second field prediction, then a motion vector ofa macro block having the highest parameter X is selected.
 14. A motionvector conversion method according to claim 13, wherein the parameter Xis a parameter selected from the group consisting of the number ofpixels in each of portions of one or more macro blocks in said P frameto be dropped, which portions overlap with a macro block of a motionvector in the P frame next to said P frame to be dropped in order to bereferred to; a value obtained by dividing the number of pixels in eachof portions of one or more macro blocks in said P frame to be dropped,which portions overlap with a macro block of a motion vector in the Pframe next to said P frame to be dropped in order to be referred to, bya macro block bit number; a value obtained by dividing the number ofpixels in each of portions of one or more macro blocks in said P frameto be dropped, which portions overlap with a macro block of a motionvector in the P frame next to said P frame to be dropped in order to bereferred to, by a macro block quantization scale; and a value obtainedby dividing the number of pixels in each of portions of one or moremacro blocks in said P frame to be dropped, which portions overlap witha macro block of a motion vector in the P frame next to said P frame tobe dropped in order to be referred to, by the product of a macro blockquantization scale and the macro block bit number.
 15. A motion vectorconversion apparatus for an image information conversion apparatus towhich a bit stream representative of interlaced scanned imagecompression information of MPEG2 is inputted and from which a bit streamrepresentative of progressively scanned image compression information ofMPEG4 is outputted, wherein 16×16 motion vectors of MPEG2 of theinputted bit stream representative of image compression information ofMPEG2 are inputted and 8×8 motion vectors of MPEG4 and 16×16 motionvectors of MPEG4 are successively produced, said motion vectorconversion apparatus comprising: dropping means for dropping every otherone of I frames and P frames of the inputted bit stream of MPEG2 andsupplying 16×16 motion vectors of MPEG2 of the remaining I frames and Pframes to said motion vector production means so that a bit stream ofMPEG4 of a reduced frame rate and a low bit rate may be produced by saidmotion vector production means.
 16. A motion vector conversion apparatusaccording to claim 15, wherein said apparatus serves as motion vectormodification means; and said apparatus further comprises: a buffer forstoring motion vector information and macro block information in theinputted bit stream representative of image compression information ofMPEG2 in each P frame to be dropped; determination means fordiscriminating a motion vector to be added in said P frame to be droppedbased on the information stored in said buffer; and addition means foradding the motion vector determined by said determination means, toproduce an 8×8 motion vector.
 17. A motion vector conversion apparatusaccording to claim 16, wherein letting a parameter X be taken as aparameter associated with each motion vector in said P frame to bedropped, a motion vector having the highest parameter X in said P frameto be dropped is determined as said motion vector to be added in said Pframe to be dropped.
 18. A motion vector conversion apparatus accordingto claim 17, wherein the parameter X is a parameter selected from thegroup consisting of the number of pixels in each of portions of one ormore macro blocks in said P frame to be dropped, which portions overlapwith a macro block of a motion vector in the P frame next to said Pframe to be dropped in order to be referred to; a value obtained bydividing the number of pixels in each of portions of one or more macroblocks in said P frame to be dropped, which portions overlap with amacro block of a motion vector of the P frame next to said P frame to bedropped in order to be referred to, by a macro block bit number; a valueobtained by dividing the number of pixels in each of portions of one ormore macro blocks in said P frame to be dropped, which portions overlapwith a macro block of a motion vector of the P frame next to said Pframe to be dropped in order to be referred to, by a macro blockquantization scale; and a value obtained by dividing the number ofpixels in each of portions of one or more macro blocks in said P frameto be dropped, which portions overlap with a macro block of a motionvector in the P frame next to said P frame to be dropped in order to bereferred to, by the product of a macro block quantization scale and themacro block bit number.
 19. A motion vector conversion apparatusaccording to claim 16, wherein letting a parameter X be taken as aparameter associated with each motion vector in said P frame to bedropped, a motion vector having the lowest parameter X in said P frameto be dropped is determined as said motion vector to be added in said Pframe to be dropped.
 20. A motion vector conversion apparatus accordingto claim 19, wherein the parameter X is a parameter selected from thegroup consisting of a macro block bit number in each of portions of oneor more macro blocks in said P frame to be dropped, which portionsoverlap with a macro block of a motion vector in the P frame next tosaid P frame to be dropped in order to be referred to; a macro blockquantization scale in each of portions of one or more macro blocks insaid P frame to be dropped, which portions overlap with a macro block ofa motion vector in the P frame next to said P frame to be dropped inorder to be referred to; and the product of a macro block quantizationscale and the macro block bit number in each of portions of one or moremacro blocks in said P frame to be dropped, which portions overlap witha macro block of a motion vector of the P frame next to said P frame tobe dropped in order to be referred to.
 21. A motion vector conversionapparatus according to claim 16, wherein in the case where any of one ormore macro blocks in said frame to be dropped, which overlap with amacro block of a motion vector in the P frame next to said P frame to bedropped in order to be referred to, is different from a macro blockwhose DCT coefficients are not coded and a macro block free from motioncompensation, if all of the overlapping macro blocks in said frame to bedropped are intra-macro blocks, then a zero motion vector is selected.22. A motion vector conversion apparatus according to claim 16, whereinin the case where any of one or more macro blocks in said frame to bedropped, which overlap with a macro block of a motion vector in the Pframe next to said P frame to be dropped in order to be referred to, isdifferent from a macro block whose DCT coefficients are not coded and amacro block free from motion compensation and further the overlappingmacro blocks in said frame to be dropped contains at least one macroblock whose macro block mode is not an intra-macro block, if aprediction mode carried out for those of the overlapping macro blocks,whose macro-block mode is not an intra-macro block, is a second fieldprediction, then a motion vector of a macro block having the highestparameter X is selected.
 23. A motion vector conversion apparatusaccording to claim 22, wherein the parameter X is a parameter selectedfrom the group consisting of the number of pixels in each of portions ofone or more macro blocks in said P frame to be dropped, which portionsoverlap with a macro block of a motion vector in the P frame next tosaid P frame to be dropped in order to be referred to; a value obtainedby dividing the number of pixels in each of portions of one or moremacro blocks in said P frame to be dropped, which portions overlap witha macro block of a motion vector in the P frame next to said P frame tobe dropped in order to be referred to, by a macro block bit number; avalue obtained by dividing the number of pixels in each of portions ofone or more macro blocks in said P frame to be dropped, which portionsoverlap with a macro block of a motion vector in the P frame next tosaid P frame to be dropped in order to be referred to, by a macro blockquantization scale; and a value obtained by dividing the number ofpixels in each of portions of one or more macro blocks in said P frameto be dropped, which portions overlap with a macro block of a motionvector in the P frame next to said P frame to be dropped in order to bereferred to, by the product of a macro block quantization scale and themacro block bit number.
 24. A motion vector conversion apparatusaccording to claim 15, wherein said motion vector conversion apparatusserves as motion vector modification means; and a motion vector in the Pframe next to said I frame to be dropped is extended twice in thetemporal direction, based on motion vector information and macro blockinformation in the inputted bit stream representative of imagecompression information of MPEG2, to produce an 8×8 motion vector.
 25. Amotion vector conversion apparatus according to claim 15, wherein saidmotion vector conversion apparatus serves as motion vector modificationmeans; and an 8×8 motion vector of MPEG4 is obtained from a motionvector of MPEG2 based on motion vector information in the inputted bitstream representative of image compression information of MPEG2 by saidmotion vector conversion apparatus and is inputted; and a re-search fora motion vector centered at the inputted 8×8 motion vector value isperformed to modify the motion vector such that one of four 8×8 motionvectors of MPEG4 cooperatively forming one macro block, which has thesmallest prediction residual, is allocated to a 16×16 motion vector, toproduce the 16×16 motion vector.
 26. A motion vector conversionapparatus according to claim 15, wherein in the case where any of one ormore macro blocks in said frame to be dropped, which overlap with amacro block of a motion vector in the P frame next to said P frame to bedropped in order to be referred to, is different from a macro blockwhose DCT coefficients are not coded and a macro block free from motioncompensation, if all of the overlapping macro blocks in said frame to bedropped are intra-macro blocks, then a zero motion vector is selected.27. A motion vector conversion apparatus according to claim 15, whereinin the case where any of one or more macro blocks in said frame to bedropped, which overlap with a macro block of a motion vector of the Pframe next to said P frame to be dropped in order to be referred to, isdifferent from a macro block whose DCT coefficients are not coded and amacro block free from motion compensation and further the overlappingmacro blocks in said frame to be dropped contains at least one macroblock whose macro block mode is not an intra-macro block, if aprediction mode carried out for those of the overlapping macro blocks,whose macro-block mode is not an intra-macro block, is a second fieldprediction, then a motion vector of a macro block having the highestparameter X is selected.
 28. A motion vector conversion apparatusaccording to claim 27, wherein the parameter X is a parameter selectedfrom the group consisting of the number of pixels in each of portions ofone or more macro blocks in said P frame to be dropped, which portionsoverlap with a macro block of a motion vector of the P frame next tosaid P frame to be dropped in order to be referred to; a value obtainedby dividing the number of pixels in each of portions of one or moremacro blocks in said P frame to be dropped, which portions overlap witha macro block of a motion vector in the P frame next to said P frame tobe dropped in order to be referred to, by a macro block bit number; avalue obtained by dividing the number of pixels in each of portions ofone or more macro blocks in said P frame to be dropped, which portionsoverlap with a macro block of a motion vector in the P frame next tosaid P frame to be dropped in order to be referred to, by a macro blockquantization scale; and a value obtained by dividing the number ofpixels in each of portions of one or more macro blocks in said P frameto be dropped, which portions overlap with a macro block of a motionvector of the P frame next to said P frame to be dropped in order to bereferred to, by the product of a macro block quantization scale and themacro block bit number.